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Claims 

What is claimed is: 

1. A method comprising the steps of: 

receiving rating information associated with a document from one or more evaluators; 
receiving a signal relevant to a criteria; and 

determining whether to deliver the document in response to the signal based on the 
criteria and the rating information from the one or more evaluators. 

2. The method of claim 1, further comprising: 

selecting a plurality of evaluators to rate the document; and 
passing the document to the plurality of evaluators for rating. 

3. The method of claim 1, wherein the signal is a request received from an entity, the entity 
is associated with a suitability standard, and the determining action comprises determining 
whether the document satisfies the suitability standard. 

4. The method of claim 1, further comprising: 

determining an aggregate rating based on the rating information, wherein the action of 
determining whether to deliver the document is further based on the aggregate rating of the 
document. 

5. The method of claim 3, further comprising: 

determining an aggregate rating based on the rating information; and 
selecting the electronic document based on the aggregate rating and the suitability 
standard. 

6. The method of claim 3, further comprising: 
storing the suitability standard in a database; and 
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processing the database to determine the suitability standard. 

7. The method of claim 1 , further comprising: 

identifying at least one trust score, wherein each trust score is associated with a specific 
one of a plurality of e valuators; 

determining an aggregate rating based on the rating information and the at least one trust 
score, wherein determining whether to deliver the document is further based on the aggregate 
rating. 

8. The method of claim 7, wherein at least one trust score is based on the geographical 
location of the associated specific one of the plurality of evaluators. 

9. The method of claim 7, wherein at least one trust score is based on one or more prior 
content ratings received from the associated specific one of the plurality of evaluators. 

10. The method of claim 7, wherein at least one trust score is based on an industry associated 
with the associated specific one of the plurality of evaluators. 

1 1 . The method of claim 7, wherein at least one trust score is based on a rating deviation of 
an evaluator, wherein the rating deviation is based on a comparison of (i) rating information for 
one or more documents received from the evaluator and (ii) rating information for the one or 
more documents received from one or more other evaluators. 

12. The method of claim 7, wherein at least one trust score is based on a rating deviation of 
an evaluator, wherein the rating deviation is based on a comparison of (i) rating information for 
one or more documents received from the evaluator and (ii) aggregate ratings for the one or more 
documents. 

13. The method of claim 7, wherein the trust score of an evaluator is based on rating 
information previously received from the evaluator for one or more documents. 
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14. The method of claim 7, further comprising: 

determining one or more revised trust scores for one or more of the plurality of 
evaluators; and 

determining a revised aggregate rating based on the one or more revised trust scores. 

15. The method of claim 7, wherein at least one of the plurality of trust scores is based on at 
least one of the geographical location of the associated evaluator and prior rating information 
received from the evaluator. 

16. The method of claim 1, wherein the document is an advertisement. 

17. The method of claim 1, wherein the document comprises at least one of a web page, 
content that can be used in a web page, and a program. 

18. The method of claim 1, further comprising the action of selecting a plurality of evaluators 
to evaluate the document. 

19. The method of claim 18, wherein the plurality of evaluators are selected using a random 
selection algorithm. 

20. The method of claim 1, wherein the criteria comprises a sensitivity score. 

21 . The method of claim 1, further comprising 
receiving new rating information for the document; and 

processing the new rating information to determine a revised rating associated with the 
document. 

22. The method of claim 1, wherein the aggregate rating comprises one or more subject 
ratings, each associated with an evaluation criterion. 

23. The method of claim 22, wherein the evaluation criterion comprises at least one of sexual 
content, violent content, adult content, and targeted age. 



32 



PATENT 

Attorney Docket No.: 64557.000020 
GP-254-00-US 

24. The method of claim 1, wherein the aggregate rating comprises a quantity. 

25. The method of claim 1, wherein the aggregate rating comprises a mean of the rating 
information. 

26. The method of claim 1, wherein the aggregate rating comprises a mode of the rating 
information. 

27. The method of claim 1, wherein the aggregate rating comprises a median of the rating 
information. 

28. The method of claim 1, wherein the request is received by at least one of a content 
provider and a user. 

29. A computer-readable medium encoded with computer program code to rate an electronic 
document, the program code effective to perform the following: 

receive rating information associated with a document from one or more evaluators; 
receive a signal relevant to a criteria; and 

determine whether to deliver the document in response to the signal based on the criteria 
and the rating information from the one or more evaluators. 

30. A system comprising: 

rating receiving means for receiving rating information associated with a document from 

one or more evaluators; 

signal receiving means for receiving a signal relevant to a criteria; and 

determination means for determining whether to deliver the document in response to the 

signal based on the criteria and the rating information from the one or more evaluators. 

31. A system comprising: 
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a rating receiving device that receives rating information associated with a document 

from one or more e valuators; 

a signal receiving device that receives a signal relevant to a criteria; and 

a determination device that determines whether to deliver the document in response to the 

signal based on the criteria and the rating information from the one or more evaluators. 

32. A method comprising the steps of: 

receiving rating information for an electronic document from a plurality of evaluators, 
wherein at least one evaluator is associated with a trust score; and 

determining an aggregate content rating for the electronic document by aggregating the 
rating information from at least two of the plurality of evaluators, wherein the rating information 
for at least one evaluator is considered in association with the trust score for that evaluator. 

33. A method comprising the steps of: 

transmitting a signal to a server, the server interpreting the signal as being associated with 
a criteria for a document and determining whether a document is to be delivered based in part on 
rating information received from a plurality of evaluators for one or more documents and 
whether the one or more documents relate to the criteria; and 

receiving a document in response to the signal. 

34. A method comprising: 

receiving one or more documents from a server system; and 

providing rating information to the server system for use by the server system in 

determining whether to deliver the one or more documents in response to requests for the one or 

more documents based on the rating provided. 

35. A method comprising the steps of: 
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receiving content rating information associated with an advertisement from a plurality of 
evaluators, wherein at least one of the plurality of advertisers is a partner advertisement listings 
provider; 

determining an aggregate rating for the advertisement based on the rating information; 
receiving a request for electronic delivery of one or more advertisements that satisfy ad 
content criteria; and 

determining whether to deliver the first advertisement in response to the request based on 
the ad content criteria and the aggregate rating. 
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